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Abstract — The problem of optimizing a linear objective func- 
tion, given a number of linear constraints has been a long 
standing problem ever since the times of Kantorovich, Dantzig 
and von Neuman. These developments have been followed by a 
different approach pioneered by Khachiyan and Karmarkar. 

In this paper we present an entirely new method for solving 
an old optimization problem in a novel manner, a technique 
that reduces the dimension of the problem step by step and 
interestingly is recursive. A theorem which proves the correctness 
of the approach is given. 

The method can be extended to other types of optimization 
problems in convex space, e.g. for solving a linear optimization 
problem subject to nonlinear constraints in a convex region. 

Index Terms — linear programming, optimization, dimension 
reduction. 

I. Introduction 

THE problem of optimizing a linear functional subject 
to a set of linear constraints (the so called Linear 
Programming or LP problem) has attracted many researchers; 
the first fundamental contributions to the LP problem was 
done by Kantorovich [1] and Dantzig [2], who first discovered 
the Simplex method, which is essentially a search method. 
After many years, the next break through came through 
Kachiyan [3] who proved that the problem can be solved, in 
theory, in polynomial time and then subsequently Karmarkar 
[4] discovered a method of search involving points inside 
the feasible space. Another proposed method of tackling the 
problem was the gravitational method [5]. However, in spite 
of all these many developments [6,7] the LP problem did not 
permit an easy resolution and even now a satisfactory solution 
to the problem is yet to be had. The mathematician Stephen 
Smale considers it as one of his 18 unsolved problems in 
mathematics. 

In this pape|^ we present an alternative approach to the 
solution of the LP problem, the method has the advantage that 
an LP problem of n dimensions (excluding "slack" variables) 
involving an objective function of n variables, is reduced 
to another LP problem in n-1 dimensions and so on. We 
show and prove by a rigorous theorem, that by invoking the 
convexity properties of the feasible region this stage by stage 
dimensional reduction of the problem is made possible. 

The next section gives the details of the method. 

II. Description OF Proposed Method 

The LP problem that will be dealt in this paper is concerned 
with the task of maximizing the objective function Z defined 
as: 
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Z = diXi+d2X2 + dnXn (1) 

subject to the constraints 

Ax<r (2) 

where A is a m x n matrix and x is a n x 1 variable vector 
and r is a m X 1 constant vector Thus we are optimizing an 
objective function Z, involving n variables (dimensions), given 
m constraint planes. 

As is well known the feasible region will be a convex 
polytope whose boundaries are the constraint planes. The 
LP problem then consists of searching for that point x, in 
the feasible region which has the maximum value of Z. 
Several facts about the LP problem are well known and bears 
repetition: (i) The optimum point if it exists will be a vertex 
and lie on the boundary, (ii) the optimum vertex will be at 
the intersection of at least n boundary planes, (iii) the feasible 
region is a convex region, by which if there are any two points 
which are in the feasible region then the line joining P and Q 
will also lie in the feasible region and (iv) the last statement 
will be true even if P and Q are on the boundary of two planes, 
if this happens, then the line PQ then will either be entirely 
inside the feasible region or lie on the boundary. 

In the following it is assumed that an optimum vertex to 
the chosen LP problem exists. Before, we proceed further it 
is necessary to define some terms: 

We will call those planes whose intersections constitute the 
optimum vertex as "roof" planes, from (ii) above, we can 
see that there must obviously be at least n roof planes. We 
define Ud as the unit vector which points in the optimum 
direction, that is n^j will have components proportional to 
{di,d2,d3, ..dn}- Similarly we define Uk as the unit vector 
which is the outward normal of the /c*'* boundary plane, in 
this case Uk is defined as that vector whose components are 
proportional to the coefficients of the fc*''row of the matrix A, 
hence is proportional to the vector {aj-i, a/j2, ats, ••afcn}- 
It is assumed that all the normal vectors are made to point 
outwards, away from the feasible region. 

Let us define the the angle 9^ by the dot product relationship 
cos{9k) — rLk-Tid- We now define the "flattest plane" as that 
plane k which is such that 9k < 9j, for (j — l,2,...,m). 
Obviously, if k is the flattest plane then Uk-rid > rij.Ud for 
(j = l,2,...,m) 

For the sake of our argument, let us assume that the 
optimum direction is "upwards", (there is no loss of generality 
in this assumption). Now we make a crucial observation: if 
the problem has a single (unique) solution, then the optimum 
vertex will lie on the plane which is the "flattest". This 
observation follows from the fact that the feasible region is 
convex, and because of this the flattest plane, must be one of 



2 



the "roof" planes, see figure. In fact, if the flattest plane is 
NOT one of the roof planes, that is if a steeper feasible plane 
is "above" the flattest plane, then the feasible region cannot be 
convex - see figure. The conclusion is true for 2-dimensions,3- 
dimensions and for n-dimensions see figure. We give a formal 
proof in Section V. 

From the above observation we can buUd an algorithm 
which is described: 

1) We start with a properly defined LP problem such that 
all the constraints are in the form given in Eq. (2), we then 
calculate all the outward normals nj,j = 1,2, ...,m of each 
of the m constraint planes. 

2) We take dot products of all the normals with the object 
function direction i.e. we find all nj.nd,,] = 1,2,...,to. By 
examining each such dot product we identify the "flattest" 
plane. If k is the flattest plane then it will have the property 
nk-rid ^ nj.Ud for all (j = 1, 2, m). 

3) Now since k is the flattest plane it must contain the 
optimum vertex. We then examine all the coefficients of the 
fc*'* plane and choose that coefficient which is the largest (say 
dkn), we then use the kth inequality as an equation, and get 
an expression for a;„, in terms of xi, X2,—;Xn-i- 

4) We substitute for .T„using the above expression in all the 
other constraint planes and delete the fc*'' plane. We will have 
m-1 constraint planes each of which is a function of only 
n-1 variables. Next we substitute for Xn in the expression 
for the objective function Z, the new Z will be a linear 
function of only n-1 variables xi, X2,.—,Xn-i- (However, 
we need to retain the equation of the deleted fc*'* plane, for 
backsubstitution later on). 

The idea behind elimination is simple: Since the optimum 
vertex will be on the flattest plane, future searches need 
be conducted only in this plane i.e. n-1 dimension space. 
Eliminating one variable by using the equation of this plane 
ensures that a search is conducted in this plane, but with a 
new objective function which does not have this variable. 

5) The objective function and the m-1 constraints obtained 
in step 4 represents a LP problem of reduced dimensions. 
We now go back to step 2 and reduce the problem to n-2 
dimensions and so on... 

6) After we have recursively reduced the problem to a single 
variable, say xi, we find out that value of Xi which maximizes 
the objective function Z which is now a fimction of 

this single variable and which satisfies all the constraints. 

7) Having found xi the rest of the variables .T2,....,.t„ can 
be found by backsubstituting in the equations representing 
the planes which were used for the elimination of variables, 
starting from the last plane and proceeding to the first in 
reverse order. 

8) The value of x\, X2,....,Xn, finally obtained represents 
the coordinates of the optimum vertex. 

The number of steps in the reduction from an n-dimesion 
problem to 1 is n-1, however there is a word of caution: Every 
time we have finish task 2), we must ensure that the current 
flattest plane is NOT a redundant plane, by the latter we mean 
a plane which is entirely outside the feasible region. In case 
the current flattest plane is a redundant plane then it must 



be deleted and the next flattest plane should be chosen (after 
testing that it is not redundant) 

III. Details of Method: 

In this section we briefly write down the various steps 
involved in the method. 

Denoting the objective function vector d and the vector of 
the fundamental variables x as: 

d = {di,d2,d3,..dn}, (3) 

X = {xi,X2,X3,..Xn}, (4) 

then the LP problem involves the task of finding out the 
optimum valueV defined as the maximum value of Z, where 
Z = d.x i.e 

V = Max{Z) (5) 
subject to the constraints : 

oiia;i -I- ai2a;2 -I- 013X3 -h .. -I- ai„a;„ < ri 

02ia;i -I- a22X2 + a23X3 + ■■ + a2nXn < r2, 

(6) 

ClmlXl + am2X2 + am3X3 + ■■ + dmnXn < Tm 

Normalization: 

It will be assumed that in the above equations the ob- 
jective function vector d is a unit vector and that each row 
of the constraint equations have been normalized, so that 
the squares of the coefficients sum up to unity. If this has 
not been done (say for the jth row) one can evaluate N j 

where Nj = ^a'j-^ + + ••• + and then divide the j*'' 
constraint by Nj and redefining aji/Nj by the coefficient aji 
for each i = l,2..,n and the constant rj by rj/Nj. This 
procedure is adopted for convenience because then the array 
{aji,aj2,aj3, ..ajn} become the components of the normal 
vector Tij to the j*'^ constraint plane. 
Reduction procedure: 

We now demonstrate the dimension of the LP problem is 

reduced step by step. 

(i) Find flattest plane: 

calculate tr = cos{6r) for all r = (1, 2, ..m) as: 

n 
i=l 

(ii) Find that plane k, such that 

tk >tr (r = l,2,....,m) (8) 

the constraint plane k then will be the flattest plane and 
therefore will contain the optimum vertex. Since the optimum 
vertex at the intersection of n planes, it is necessary to find the 
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Other n-1 planes chosen out of m, whose common intersection 
point is the optimum vertex. Now since we know that the 
optimum point lies on this plane k we enforce this latter 
condition, by using the A"' constraint as an equation and 
then eliminating one of the variables {xi,X2,Xs, ..Xn}- The 
variable to be eliminated will be that which has the largest 
coefficient in the fc'^'equation - to reduce round-off errors. 

At this point it is assumed, for the argument, that the kth 
plane has been tested for nonredundancy, then the algorithm 
proceeds to step (iii). (If the plane is redundant, see Appendix, 
then it must be deleted and the next flattest but not redundant 
plane must be chosen). 

(iii)The k*^ equation is 

akixi + ak2X2 + ak3X3 + .. + a^nXn - rfe = (9) 

find the coeficient with the largest magnitude, say, it is the 
that is 



dj-i ^ {dj-i - dj ^MOl) 



O-kj 



dj ^ 



\ak] \ > Wktl {i = 1,2. .n) 

the above inequalities indicate that akj is the largest coef- 
ficient in the k*'^ constraint plane , so the variable Xj can be 
ehminated from all the rest of the inequaUties and objective 
function, since 

we can write the fc"* constraint equation as: 



dn ^ {dn 



dj—) 

O-kj 



and we have a constant term dowhich is initiaUy zero, 
becomes nonzero 



(10) 



_ fk _ O-kl _ ak2 _ Clkn 
Xj — X\ X2 ' ' • »^7i 

^kj ^kj ^kj ^kj 

(iv) Now we will substitute (10) for variable Xj wherever it 
occurs in all the other constraints, typically if we substitute it 
in the m*'' constraint equation, the coefficients will be redefined 
and it will not have the Xj. Hence, we then have (for this 
u*'' equation): 



ak2 N 
0-kj 



(11) 



^ 



(r„ - a. 



UJ ) 

O-kj 



Note o„j = 0. The above replacements are done for con- 
straints u (u=l,2, m), except k which is (saved in memory), 

but deleted and not considered further consideration in the 
reduction process. 

(v) The new objective fimction 

After eliminating Xj the ncoefficients of the objective 
function become: 



di -It- {di — dj 



d2 ^ {d2 - dj 



a/ci > 

O-kj ' 

ak2 > 
akj ' 



do {do + dj —) 

O-kj 



Z = do+diXi+d2X2 + ..- + dj-iXj-i+dj+iXj+i... + ..dnXn 

(12) 

It may be noticed that the objective fiinction has one less 
varaiable: Xj is not present. 

(vi) The equation (12) now needs to be maximized with the 
m-1 constraints given by equations (11) for all planes u except 
k. Hence (12) and (11) now represent an LP problem but with 
one less dimension i.e. n-1. 

Hence, we normalize the set of equations (11) and (12) as 
stated above. Calculate the outward normals Uj of the new 
constraint planes, for which we need one feasible point and 
then we put all the constraints in the standard form (viz.eqs (4) 
to (6)) and proceed to paragraph (i) and find the new flattest 
plane in n-1 dimensions wrt the new objective function. 

In this manner, the problem is reduced to a single variable 
involving an objective function of a single variable say x\ 
which can be maximized within the constraints. 

Having, found xi , then the previous plane which was 
just eliminated, is read from storage, this is an equation 
in two variables, it will have xi and another variable, say 
X2, using this X2 is found. After this the plane which was 
ehminated second-last is retrieved from storage, this contains 
an additional variable say 2:3 which is then found. This back 
substitution process which is nothing but a version of Gauss- 
Seidel elimination process, is used to find the coordinates of 
the optimum vertex namely {xi,X2, x^, ..a;„). 
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IV. Figures 



Polytope in 2D/3D 

We see that w/ith respect to the 
objective function direction, the "flattest" 
plane containBthe Dptimum point P. 
PQ.and PR are roof planes. 




Fig la 

in all the figures the optimum direction is 




n-dimensionai poivtop^ (a figurative 

Lt't ABand BC be two planes, let ushypotheBke that AB is. the "flattest" plane and not the roof. W@ 
demonSitrate thatthisi cannot happen: Let BC be another plane intersecting AB- and steeper. Let us 
^Etume AB and are feasible boundaries. Using the fart that the feasible space is convex, we argue that 
since Pand Q are feasible points the line fO, must be entirely in the feasible region. This is obviously not 
true. Hence, the situation such as the above cannot arise - the flattest plane must be one of the roof 
planes. 

To briefly describe the worldng of the algorithm, it is 
perhaps worthwhile to say that as we reduce the dimensions 
step by step we will be choosing the "flattest" plane with 
respect to the present optimum direction. For instance, suppose 
we are now in the r+1 th step, that is we are now searching in 
n-r dimensional space, we will find that in this space some of 
the planes which were having a feasible region in the previous 
n-r+1 dimension space, may now become redundant, i.e. they 
will lie out side the present feasible region. Of course, it is 
not necessary for us to know which one unless it happens to 
be the "flattest", in the latter case it has to be eliminated. In 
other words in the algorithm we need to test only those planes 
for redundancy, which have currently qualified as "flattest". 
Though it is acknowledged that the test for redundancy, is a 
bit of a down side, the dimension reduction along with the 
discovery of a roof plane at each step registers a plus score 
for the algorithm. After all there are only n roof planes to be 
discovered and one is being discovered in each cycle (step). 

It is quite possible that the method may be extendable to 
other types of optimization problems, for instance if one is to 
deal with the case of optimizing a linear objective function, but 
under nonlinear (but convex) constraints which still maintain 
a convex feasible region, then this method is useful. Because 
such constraint surfaces can be approximated by a piece-wise 
patchwork of planes. 



V. Theorem 

Theorem: The optimum vertex, if it exists lies on the flattest 
plane. 

Proof: 

We assume that the flattest plane, in the collection of m 
planes given in Eq. (2) is A, and given by the equation: 

akixi + ak2X2 + aksxs + ■■ + afc„a;„ = 

We now show that there cannot be another plane, say B, 
which is "steeper" than A and is a roof plane and contains a 
point, Q, whose optimum value Z(Q), is higher than Z(P) of 
a point P, on A. Let us assume, for the sake of argument such 
a B actually exists and is given by the equation: 

QjiXi + aj2X2 + ajsXs + ..+ ajnXn = rj 

as the proof proceeds it will be shown that such a B cannot 
exist. 

Now since A and B are non-parallel they will "intersect" in 
a region R. (For the case when n=2, R is a single point, and 
if n=3, R is a line and if n=4, R is a 2-d plane and so on). 

We will now perform the following coordinate transforma- 
tions X —i' x' : 

(i) We first shift the origin to some point in the region R, 

(ii) We then rotate the coordinate system such that X2 
is along the optimum direction, and the coordinate x[ is 
perpendicular to it as shown, the other n-2 coordinate axis 
will be (aig, X4, .., a;^). We will be able to write down the 
equations for A and B in terms of a new set of coefficients 
such as: 

Plane A: 

sin{a)xi — cos{a)x2 + 030:3 + .. + a'„x'^ = 
Plane B: 

sin{l3)x[ - cos{l3)x'2 + + .. + b'^Xi = 
Note since we have chosen the new origin to be in the region 
R, the constant terms in the r.h.s. of the above equations are 
zero; as the origin is assumed, by our choice, to lie on both 
planes. 

Also since we have assumed that Plane B is steeper than 
plane A w.r.t. the optimum direction X2, we must have: 

P > a 

Now we show that the above inequality is untenable, with 
the condition that the optimum value lies on B rather than A. 

Now convexity implies that for every feasible point P which 
lies on A and another feasible point Q which lies on B, the 
line segment PQ must be in the feasible region. We will now 
show that this cannot happen for the planes A and B chosen 
as above. To prove the latter sentence all we need is to choose 
two feasible points P, Q lying on A and B respectively, and 
show that the line PQ cannot be in the feasible region. 

We choose P as follows: 

Let P be the point whose coordinate is (xp, yp, 0, 0, 0), 
that is we have chosen, x[ = xp^x^ = yp,x'^ — 
0,2:4 = 0, ..jxjj = 0. Similarly we choose Q as the point 
with coordinate (xq, yg, 0, 0, 0), that is we have chosen 
x[ — XQ,x'2 = VQjx'^ =0,0:4 — 0, ..jxjj = 0. Substituting 
these two coordinates in the equation for their corresponding 
planes we have the conditions , for P and Q to lie on A and 
B respectively as: 
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sin{a) Xp — cos{a) yp = 
sin{j3)xQ - cos{(3)yQ = 



(13) 
(14) 



In order to draw a 2-dimension figure for an n-dimensional 
situation, we do as follows: consider a planar section contain- 
ing the origin O and the {x'i,x'2) axis, in this figure, P and 
Q will appear as points lying on lines OA and OB, which 
represent the respective planes, A and B. Now the value of 
the objective function Z at points P and Q are Z{P) = yp 
and Z{Q) = yq, since /3 > a, we can see from the figure 
that yq > yp, i.e. Z{Q) > Z{P) , but the line segment 
PQ is outside the feasible region. Hence, plane B, having the 
property as above, cannot exist, thus the theorem is proved 
QED. 



Un-feasible Region 
A 




Fig. 2 



VI. A BRIEF ON REDUNDANT PLANES 

In the description of the present method, we had said that the 
number of steps in reducing the problem from n dimensions to 
1 would be n-1, however much depends on the actual geometry 
of the polytopes. It must be ensured that the current flattest 
plane is not a redundant plane, that is, it should not be a 
plane completely "above" the feasible region, if such a thing 
happens then obviously the optimum vertex point cannot lie on 
it and then there is no sense using the equation to the flattest 
plane for elimination of a variable/reduction - the redundant 
plane must simply be deleted and the next "flattest" plane 
must be found. The Linear Programming literature contains a 
number of techniques of detecting redundant planes and these 
techniques may be used. 

Since we need to only test one plane at a time, the current 
flattest plane to ensure that it is not redundant, perhaps, the 
simplest way to begin is to use Monte Carlo techniques 
(e.g. see [8]). That is randomly generate coordinates of many 
feasible points inside the polytope and then from each of these 
points "draw" straight lines in the optimum direction, all of 
them will intersect some plane or the other and out of these 
a few will hit the "flattest" plane (see Figure 2). The points 
where the flattest plane is hit will become feasible points in 
the n-1 dimension space upon dimension reduction (one of 
the feasible points can also be used to calculate the outward 
normals of the new system of planes). 

If it so happens that the flattest plane is redundant then the 
plane will not be hit before another "lower" plane, proving that 
the plane is redundant and can be deleted. The diagram in Fig 
3, shows LM as the "flattest" but redundant plane, hence this 



will be deleted by the algorithm and the next "flattest" plane 
PQ will be retained. It is not necessary to remove all redundant 
planes, but only those which happen to qualify as "flattest" at 
any stage. 

This method has the advantage that it can be easily imple- 
mented by an algorithm which is parallelizable, thus one can 
very efficiently use multiple processors. 



One of the ways to find if the "flattest" plane is not a 
redundant plane istogenerate feasibe points at randoi 
(sav) A,B,C,D,E,F... and draw lines from themuerticallv 
(in the optimum direction). If the "flattest" plane PQi5 




ant th 



1 atle 



!etthe plane at feasible points c,i 
isa redundant plane as it is 
le feasible region. 



VII. Conclusions 

A new method of solving the LP problem which is done by 
dimensional reduction and is aesthetically pleasing has been 
found. This paper also shows that there exists a recursive 
method of solving the LP problem, a fact which was not 
known and therfore novel. The possible utility of the strategy 
described in this paper for dealing with other types of opti- 
mization problems involving nonlinear constraints have also 
been briefly indicated. 
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